Deep Recurrent Neural Network Research Articles

Design and Experimental Validation of Deep Reinforcement Learning-Based Fast Trajectory Planning and Control for Mobile Robot in Unknown Environment.

This article is concerned with the problem of planning optimal maneuver trajectories and guiding the mobile robot toward target positions in uncertain environments for exploration purposes. A hierarchical deep learning-based control framework is proposed which consists of an upper level motion planning layer and a lower level waypoint tracking layer. In the motion planning phase, a recurrent deep neural network (RDNN)-based algorithm is adopted to predict the optimal maneuver profiles for the mobile robot. This approach is built upon a recently proposed idea of using deep neural networks (DNNs) to approximate the optimal motion trajectories, which has been validated that a fast approximation performance can be achieved. To further enhance the network prediction performance, a recurrent network model capable of fully exploiting the inherent relationship between preoptimized system state and control pairs is advocated. In the lower level, a deep reinforcement learning (DRL)-based collision-free control algorithm is established to achieve the waypoint tracking task in an uncertain environment (e.g., the existence of unexpected obstacles). Since this approach allows the control policy to directly learn from human demonstration data, the time required by the training process can be significantly reduced. Moreover, a noisy prioritized experience replay (PER) algorithm is proposed to improve the exploring rate of control policy. The effectiveness of applying the proposed deep learning-based control is validated by executing a number of simulation and experimental case studies. The simulation result shows that the proposed DRL method outperforms the vanilla PER algorithm in terms of training speed. Experimental videos are also uploaded, and the corresponding results confirm that the proposed strategy is able to fulfill the autonomous exploration mission with improved motion planning performance, enhanced collision avoidance ability, and less training time.

Comparison of time series temperature prediction with auto-regressive integrated moving average and recurrent neural network

The region of Beni Mellal, Morocco is heavily dependent on the agricultural sector as its primary source of income. Accurate temperature prediction in agriculture has many benefits including improved crop planning, reduced crop damage, optimized irrigation systems and more sustainable agricultural practices. By having a better understanding of the expected temperature patterns, farmers can make informed decisions on planting schedules, protect crops from extreme temperature events, and use resources more efficiently. The lack of data-driven studies in agriculture impedes the digitalization of farming and the advancement of accurate long-term temperature prediction models. This underscores the significance of research to identify the optimal machine learning models for that purpose. A 22-year time series dataset (2000-2022) is used in the study. The machine-learning model auto-regressive integrated moving average (ARIMA) and deep learning models simple recurrent neural network (SimpleRNN), gated recurrent unit (GRU), and long short-term memory (LSTM) were applied to the time series. The results are evaluated based on the mean absolute error (MAE). The findings indicate that the deep learning models outperformed the machine-learning model, with the GRU model achieving the lowest MAE.

Long Short-Term Memory and Bidirectional Long Short-Term Memory Modeling and Prediction of Hexavalent and Total Chromium Removal Capacity Kinetics of Cupressus lusitanica Bark

Hexavalent chromium [Cr(VI)] is a high-priority environmental pollutant because of its toxicity and potential to contaminate water sources. Biosorption, using low-cost biomaterials, is an emerging technology for removing pollutants from water. In this study, Long Short-Term Memory (LSTM) and bidirectional LSTM (Bi-LSTM) neural networks were used to model and predict the kinetics of the removal capacity of Cr(VI) and total chromium [Cr(T)] using Cupressus lusitanica bark (CLB) particles. The models were developed using 34 experimental kinetics datasets under various temperature, pH, particle size, and initial Cr(VI) concentration conditions. Data preprocessing via interpolation was implemented to augment the sparse time-series data. Early stopping regularization prevented overfitting, and dropout techniques enhanced model robustness. The Bi-LSTM models demonstrated a superior performance compared to the LSTM models. The inherent complexities of the process and data limitations resulted in a heavy-tailed and left-skewed residual distribution, indicating occasional deviations in the predictions of capacities obtained under extreme conditions. K-fold cross-validation demonstrated the stability of Bi-LSTM models 38 and 43, while response surfaces and validation with unseen datasets assessed their predictive accuracy and generalization capabilities. Shapley additive explanations analysis (SHAP) identified the initial Cr(VI) concentration and time as the most influential input features for the models. This study highlights the capabilities of deep recurrent neural networks in comprehending and predicting complex pollutant removal kinetic phenomena for environmental applications.

Dynamic task scheduling in edge cloud systems using deep recurrent neural networks and environment learning approaches

The rapid growth of the cloud computing landscape has created significant challenges in managing the escalating volume of data and diverse resources within the cloud environment, catering to a broad spectrum of users ranging from individuals to large corporations. Ineffectual resource allocation in cloud systems poses a threat to overall performance, necessitating the equitable distribution of resources among stakeholders to ensure profitability and customer satisfaction. This paper addresses the critical issue of resource management in cloud computing through the introduction of a Dynamic Task Scheduling with Virtual Machine allocation (DTS-VM) strategy, incorporating Edge-Cloud computing for the Internet of Things (IoT). The proposed approach begins by employing a Recurrent Neural Network (RNN) algorithm to classify user tasks into Low Priority, Mid Priority, and High Priority categories. Tasks are then assigned to Edge nodes based on their priority, optimizing efficiency through the application of the Spotted Hyena Optimization (SHO) algorithm for selecting the most suitable edge node. To address potential overloads on the edge, a Fuzzy approach evaluates offloading decisions using multiple metrics. Finally, optimal Virtual Machine allocation is achieved through the application of the Stable Matching algorithm. The seamless integration of these components ensures a dynamic and efficient allocation of resources, preventing the prolonged withholding of customer requests due to the absence of essential resources. The proposed system aims to enhance overall cloud system performance and user satisfaction while maintaining organizational profitability. The effectiveness of the DTS-VM strategy is validated through comprehensive testing and evaluation, showcasing its potential to address the challenges posed by the diverse and expanding cloud computing landscape.

Image Caption using VGG model and LSTM

Deep convolutional networks and recurrent neural networks have gained significant popularity in the field of image captioning tasks in recent times. As we all know the performance and the architecture of models are still eternal topic. We constructed the model using a new method to enhance its performance and accuracy. In our model, we make use of pretrained CNN model VGG (Visual Geometry Group) to extract image features, and learn caption sentence features using bidirectional LSTM(Long-Short-Term-Memory) which can better understand the meaning of sentences in the text. Then we combine the image features and caption features to predict captions for images. The dataset Flickr8K is used to train and test the model. Additionally, the model has the ability to produce captions that are shorter than a specified caption length. We evaluated our model with Bilingual Evaluation Understudy (BLEU) score which measures the similarity of predicted text and to the real text. After evaluation and comparison, our model is proved to be well-done on some conditions.

A survey of deep learning-driven architecture for predictive maintenance

Over the past decades, deep learning techniques have attracted increased attention from various research and industrial domains aligned with the development of Industry Internet-of-Things(IIoT). Specifically, with the advantage of data-driven methods, industrial organizations are seeking novel proactive strategies supported by analytic models to guarantee the quality of their production by observing degradation or predicting failure ahead of the occurrence of the component or asset. Predictive strategies are expected to promise the influence of unnecessary maintenance interruptions and mitigate the consequence of that, hence, extending the remaining useful life of products. This paper conducts a survey of the utilization of deep learning technologies on engineering applications where they provide satisfactory solutions with respect to specific data types or input signals. 106 primary papers are reviewed on deep learning–driven approaches which mainly explore five of the most popular architectures in the application of predictive maintenance. The main content of this paper summarizes the common advantages of each architecture and, accordingly, points out their limitations, as well as describes the application scopes of fully connected deep neural networks, convolutional neural networks, stacked autoencoders, deep belief networks, and deep recurrent neural networks. Based on the technique discussion for each of them, we intend to provide a comprehensive understanding and guidance of the appropriate usage of deep learning architectures to devise an effective predictive maintenance strategy for the scientific and industrial developers whose expertise lies in the prior domain knowledge of multi-source isomerization data. Moreover, the main content demonstrated the summarization of the decisive factor by which the incremental stages of the approaches were determined, fundamentally including the dataset specification, feature extraction, and the integration of deep learning approaches.

ODRNN: optimized deep recurrent neural networks for automatic detection of leukaemia

ODRNN: optimized deep recurrent neural networks for automatic detection of leukaemia

An efficient deep recurrent neural network for detection of cyberattacks in realistic IoT environment

An efficient deep recurrent neural network for detection of cyberattacks in realistic IoT environment

Task sub-type states decoding via group deep bidirectional recurrent neural network

Decoding brain states under different cognitive tasks from functional magnetic resonance imaging (fMRI) data has attracted great attention in the neuroimaging filed. However, the well-known temporal dependency in fMRI sequences has not been fully exploited in existing studies, due to the limited temporal-modeling capacity of the backbone machine learning algorithms and rigid training sample organization strategies upon which the brain decoding methods are built. To address these limitations, we propose a novel method for fine-grain brain state decoding, namely, group deep bidirectional recurrent neural network (Group-DBRNN) model. We first propose a training sample organization strategy that consists of a group-task sample generation module and a multiple-scale random fragment strategy (MRFS) module to collect training samples that contain rich task-relevant brain activity contrast (i.e., the comparison of neural activity patterns between different tasks) and maintain the temporal dependency. We then develop a novel decoding model by replacing the unidirectional RNNs that are widely used in existing brain state decoding studies with bidirectional stacked RNNs to better capture the temporal dependency, and by introducing a multi-task interaction layer (MTIL) module to effectively model the task-relevant brain activity contrast. Our experimental results on the Human Connectome Project task fMRI dataset (7 tasks consisting of 23 task sub-type states) show that the proposed model achieves an average decoding accuracy of 94.7% over the 23 fine-grain sub-type states. Meanwhile, our extensive interpretations of the intermediate features learned in the proposed model via visualizations and quantitative assessments of their discriminability and inter-subject alignment evidence that the proposed model can effectively capture the temporal dependency and task-relevant contrast.

Hybrid weight optimized deep learning for autism and visual sensory processing disorder detection: A heuristic strategy on medical domain

An Autism Spectrum Disorder (ASD) affected individual has several difficulties with social-emotional cues. The existing model is observed with emotional face processing in adolescents and ASD and Typical Development (TD) by utilizing various body parameters. Scanning facial expressions is the initial task, and recognizing the face’s sensitivity to different emotional expressions is the next complex task. To address this shortcoming, in this work, a new autism and visual Sensory Processing Disorder (SPD) detection model for supporting healthcare applications by processing facial expressions and sensory data of heart rate and body temperature. Here, initially, the individual data regarding facial emotions and other body parameters like heart rate and body temperature are collected from various subjects. Then, the selection of optimal features is executed by a hybrid algorithm named Density Factor-based Artificial Bee Honey Badger Optimization (DF-ABHBO), where the most essential features are attained and fed to the detection phase. The optimal feature selection is made by resolving the fitness function with constraints like correlation, data variance, and cosine similarity for inter and intra-class. Finally, the autism and visual SPD detection are performed through a Hybrid Weight Optimized Deep Neural Recurrent Network (HWODNRN), where the hyperparameter and weights of “Deep Neural Network (DNN) and Recurrent Neural Network (RNN)” are optimized with the developed DF-ABHBO technique. From the result analysis, the accuracy and F1-score rate of the offered DF-ABHBO-HWODNRN method have attained 96% and 93%. The findings obtained from the simulations of the designed system achieve better performance.

Leakage monitoring and diagnosis of LNG storage tanks with temperature sensing network integration and artificial intelligence algorithm

Temperature is the most important safety monitoring indicator for leakage diagnosis during the operation phase of liquefied natural gas (LNG) storage tanks. Timely monitoring and accurate identification of LNG leakage events are crucial for accident prevention, loss reduction, and facility safety maintenance. This study integrates artificial intelligence (AI) algorithms and temperature sensing data to achieve intelligent monitoring and diagnosis of leakage in LNG storage tanks. Firstly, a comprehensive temperature sensing network is constructed by combining numerical simulation of the temperature field and temperature sensing experiments using fiber Bragg grating sensors. Secondly, Python is used to perform linear grid interpolation and flattening on the sensing network, generating 2D temperature nephogram samples that are conducive to AI algorithm recognition. Finally, sample features are extracted using machine vision, and leakage location calculation, leakage diagnosis and leakage volume calculation are implemented with the help of machine learning algorithms, achieving satisfactory accuracy on the test set. In addition, the ConvLSTM framework is introduced for deep learning and recurrent neural network training, enabling spatiotemporal prediction of the leakage area.

Prediction of treatment response in major depressive disorder using a hybrid of convolutional recurrent deep neural networks and effective connectivity based on EEG signal.

In this study, we have developed a novel method based on deep learning and brain effective connectivity to classify responders and non-responders to selective serotonin reuptake inhibitors (SSRIs) antidepressants in major depressive disorder (MDD) patients prior to the treatment using EEG signal. The effective connectivity of 30 MDD patients was determined by analyzing their pretreatment EEG signals, which were then concatenated into delta, theta, alpha, and beta bands and transformed into images. Using these images, we then fine tuned a hybrid Convolutional Neural Network that is enhanced with bidirectional Long Short-Term Memory cells based on transfer learning. The Inception-v3, ResNet18, DenseNet121, and EfficientNet-B0 models are implemented as base models. Finally, the models are followed by BiLSTM and dense layers in order to classify responders and non-responders to SSRI treatment. Results showed that the EfficiencyNet-B0 has the highest accuracy of 98.33, followed by DensNet121, ResNet18 and Inception-v3. Therefore, a new method was proposed in this study that uses deep learning models to extract both spatial and temporal features automatically, which will improve classification results. The proposed method provides accurate identification of MDD patients who are responding, thereby reducing the cost of medical facilities and patient care.

Dust impact on photovoltaic technologies: a comparative analysis using deep recurrent neural networks

ABSTRACT Photovoltaic (PV) behavior and productivity are affected by environmental parameters such as temperature, humidity, dust, etc. This study was conducted in Sohar, Oman, six standalone PV modules, each rated at 100 W, were meticulously tested outdoors for 35 days, specifically focusing on the detrimental effects of dust. What sets this research apart is the pioneering use of a deep recurrent neural network (DRNN) to comprehensively analyze and simulate the adverse impact of dust on PV power generation. The study employed various PV module technologies, including monocrystalline, polycrystalline, and flexible monocrystalline modules. The experimental findings, which encompassed a comparative analysis of both clean and dusty PV modules over the course of a month, revealed substantial performance degradation: 30.24%, 28.94%, and 36.21% for the respective PV module technologies. Furthermore, the presence of dust not only reduced power output but also led to lower module temperatures. The results obtained through practical experiments underscore the heightened negative influence of dust on solar panels, as explained by the DRNN network, resulting in a significant decrease in energy production across all panels. In the cross-validation phase at epoch number 500, the MSE for the DRNN module was 0.0145, while in the training phase at epoch number 827, it was 0.0178. A set of performance factors has been applied to test the accuracy of the results predicted from the proposed model DRNN. The sensitivity analysis showed that the flexible monocrystalline panel (Clean) is less uncertain based on its sensitivity rate of 0.0092. It is followed by the flexible monocrystalline panel (Dusty) with a rate of 0.0176.

EDLNet: ensemble deep learning network model for automatic brain tumor classification and segmentation

The brain’s abnormal and uncontrollable cell partitioning is a severe cancer disease. The tissues around the brain or the skull induce this tumor to develop spontaneously. For the treatment of a brain tumor, surgical techniques are typically preferred. Deep learning models in the biomedical field have recently attracted a lot of attention for detecting and treating diseases. This article proposes a new Ensemble Deep Learning Network (EDLNet) model. This research uses the Modified Faster RCNN approach to classify brain MRI scan images into cancerous and non-cancerous. A deep recurrent convolutional neural network (DRCNN)-based diagnostic method for early-stage brain tumor segmentation is presented. The evaluation outcomes show that the proposed tumor classification and segmentation model’s performance accurately segments tissues from MRI images. For the analysis of the proposed model, two different publicly available datasets (D1&D2) are used. For D1 and D2 datasets, a total of 99.76% and 99.87% accuracies are achieved by the proposed model. The performance results of the proposed model are more effective than the state-of-the-art network models as per the experimental results. Communicated by Ramaswamy H. Sarma

Automatic tuning of PID controllers using deep recurrent neural networks with pruning based on tracking error

Although PID controllers are common in industry, they are often poorly tuned; especially, in uncertain environments. Modern industries, with increasing complexity, motivate us to employ new intelligent methods in order to extend PID controllers beyond their usual capabilities. In this paper, an advanced machine learning scheme is utilized to improve PID controllers; for the first time, a deep dynamic neural network is employed to tune online the parameters of the traditional PID controller in order to overcome the effects of uncertainties in the closed-loop control system. To reduce the computational burden of deep recurrent neural network, a novel structural learning technique is applied to optimize the configuration. Unlike existing pruning methods, the network is pruned based on the values of neurons and the total value of the corresponding layer. Simulation of a benchmark CSTR system demonstrate that the proposed scheme performs more efficiently compared to a shallow network tuner, in the presence of uncertainties.

A Deep Learning Framework to Identify Remedial Action Schemes Against False Data Injection Cyberattacks Targeting Smart Power Systems

This article proposes aremedial actionscheme (RAS) based on the concept of deep learning to mitigate the impacts of false data injection (FDI) cyberattacks on smart power systems. As a prerequisite of such a RAS, power system operator is being in attacker's shoe to scrutinize different scenarios of cyberattacks. In design of the RAS, long short-term memory (LSTM) cells have been integrated into a deep recurrent neural network (DRNN) to effectively process the data of an intelligent archive framework (IAF), identifying the proper reaction mechanisms. Power flow analysis has been considered to examine the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">interdependence</i> between transmission/distribution sectors to react to the cyberattacks for which similar <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pre-investigated</i> remedial actions have not been saved in the IAF. Effectiveness of the proposed RAS is validated on two IEEE transmission/distribution systems, where consequences of FDI cyberattacks are reduced by 30% in case of experiencing attacks, which are not pre-investigated by system operator.

A deep learning approach for balance optimisation of patch panel assembly line

This paper introduces an integrated methodology to enhance the efficiency of patch panel assembly lines by harnessing advanced artificial intelligence (AI) techniques. The primary objective is to minimise the total completion time of patch panels, a pivotal metric for manufacturing efficiency. The study employs the Assembly Line Production Optimisation (ALPO) model to analyze and enhance the production capacity of the bottleneck station. Existing challenges related to patch panel assembly line task distribution, process content, and tooling layout are scrutinised against production process standards for operational measurements. To achieve improvement, Deep Neural Network (DNN) and Recurrent Neural Network (RNN) models are employed in hidden layers, supplemented by deep feed-forward (DFF) layers for output, in adherence to engineering design principles. The bottleneck station's load rate is successfully reduced to below 80%, contributing to an overall balance rate increase from 73% to 88%. These outcomes signify a notable enhancement in production efficiency and employee satisfaction. The research utilises a dataset derived from the Microsoft Azure AI-based PdM dataset. The paper illustrates how AI techniques offer a comprehensive solution to address the inherent complexities in patch panel assembly line optimisation.

Statistical downscaling of maximum temperature under CMIP6 global climate models and evaluation of heat wave events using deep learning methods for Indo‐Gangetic Plain

AbstractThe rising global temperature is one of the primary concerns of the world as it impacts the economy, environment and healthcare of any country which are more pronounced a regional level. Assessment of regional impacts of climate change at a local level requires fine resolution of climate data for which a robust and fast downscaling method is needed. In this study, we use three deep learning‐based methods, namely long short‐term memory network (LSTM), deep neural network (DNN) and recurrent neural network (RNN), to downscale CMIP6 13 GCMS models data (1.25° × 1.25° resolution) global climate model (GCM) maximum temperature (Tmax) at a regional scale of 0.5° × 0.5° spatial resolution for the period 1991–2010 over the Indo‐Gangetic Plain (IGP). In addition to the temperature prediction, heat wave events have been also analysed in the study. The study found that LSTM method performs better than DNN and RNN in downscaling of all GCM model datasets when evaluated against observed maximum temperature data from the India Meteorological Department (IMD) in terms of RMSE (0.9–3.5), average of all grid MAE value between (1.2 and 2.68), correlation (0.68–0.9) along with and spatiotemporal variability. LSTM also performed better in heat wave prediction over the region with similar temporal range (12–36 events) and spatial occurrence as compared to the observation (12–28 events). Overall, the study concludes that LSTM performs better than two methods for Indo‐Gangetic Plain with best hyper parameter tuning. Hence, we propose to utilize a deep learning framework based on LSTM for downscaling GCM dataset at a finer resolution.

A model-free method based on RDPG for fiber diameter steady control

Optical fibers have made outstanding contributions to optical instruments such as micro-channel plates (MCPs) and micro-pore optics (MPOs). As the stricter performance requirements for the MCP and MPO, traditional diameter control methods are incapable. Additionally, the partial observation issue of fiber drawing process further limits the outcome of the existing control approaches. Innovative control algorithms are vital for the diameter precision enhancement. We investigate the significance of recurrent deterministic policy gradient (RDPG) on diameter control. RDPG consists of deep deterministic policy gradient (DDPG) and recurrent neural network (RNN) where DDPG is a model-free reinforcement learning algorithm for continuous control and RNN for addressing partial observation. The impact of RNN variants selection on control performance is further discussed. Results illustrate that the application of RNN substantially solves the partial observation and varying RNN variants play a positive role in diameter control, but bidirectional gated recurrent unit (BiGRU) reaches the best accuracy. Comparison experiment results indicate that RDPG is able to learn reasonable control strategies and achieves significantly more accurate and robust diameter control in contrast with the existing methods.

Multi‐objective optimization for optimal energy transporting path and energy distribution in electric vehicles energy internet

SummaryEnergy internet permits the power to stream lithely for broadcast and it aids in transporting energy to each user using electric vehicles (EVs). The energy that EVs utilize is generated using certain sources. Still, considering poor weather, the power station averts the production of energy and needs to acquire power transmitted to another location. One resolution is to set stations that aid to save surfeit energies and minimize energy loss during transportation. This paper devises a new model for choosing charging stations to charge EVs. Energy is transferred from the energy source to the charge station via bus station through optimal path using Battle Royale Jaya Optimization (BRJO) routing with fitness attributes like distance to select the shortest path and energy. The prediction of energy is done with a deep recurrent neural network (DRNN) and the charge is distributed optimally to EVs by employing a proposed Competitive Swarm‐Battle Royale Jaya Optimization (CSBRJO) by utilizing multi‐parameters, like priority, distance, predicted energy, waiting time, and EV count needed for charging. The CSBRJO outperformed with the smallest distance of 6.935 km, energy consumption of 1.676 J, path length of 5.05 m, and waiting time of 4.645 sec.